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Matrix Algebra Review 


Introduction 


This is the matrix review for A Student’s Guide to Vectors and Tensors 
(SGVT). It is not meant to be a thorough introduction to the theory and 
practice of matrix algebra, but rather a review of the matrix concepts 
you should understand when learning about vectors and tensors. 

A matrix is defined as an array of numbers. Some authors restrict the 
term “matrix” to refer only to square arrays of real numbers, but in com- 
mon usage a matrix is a rectangular (nr x n) array of complex numbers 
or “elements,” where “nr” represents the number of rows, “n” represents 
the number of columns, and “complex” means that the numbers may be 
purely real, purely imaginary, or mixed. Thus 
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are all matrices. In these examples, matrix A has “dimensions” or “or- 
der” (rows x columns) of 2x2, B of 5x1, C of 1x6, D of 4x2, E of 3x3, 
and F of 2x2. 
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If both m and n for a matrix are equal to one, that matrix is called 
a scalar (because it’s just a single value), and if either (but not both) 
of m and n are one, that matrix may be called a vector. So in the 
examples shown above, B and C are vectors; in some texts you’ll see 
B called a column vector and C called a row vector. A square matrix 
(that is, a matrix with m=n) with ones on the diagonal and zeroes in all 
off-diagonal elements (such as matrix E above) is called the “unit” or 
“identity” matrix, and a matrix with all elements equal to zero is called 
the “null” or “zero” matrix. 

Two matrices are said to be equal only if they have the same number 
of rows as well as the same number of columns and if every element in 
one matrix is equal in value to the corresponding element in the other 
matrix. Matrices are usually denoted using uppercase letters or bold font, 
often surrounded by square brackets (such as [A]), and the elements are 
often written using lowercase letters with subscripts. So you may see the 
elements of matrix [A] written as a;j , although some authors use Aij or 
[A]ij to refer to the elements of A (but be careful to note that A t j may 
also refer to the “Matrix of Cofactors” of matrix A , which you can read 
about in Section E of this document). 

So does terminology such as “row vector” mean that matrices, vectors, 
and tensors are the same thing? Not really. It’s certainly true that the 
vectors and tensors in SGVT are often represented using matrices, but 
it’s important to remember what those matrices stand for. Those arrays 
of values represent the components of vectors and tensors, and those 
components have meaning only when associated with the basis vectors 
of a particular coordinate system. Since the basis vectors are not always 
shown, it’s convenient to think of the matrix as representing the vector 
or tensor itself, and that’s fine as long as you remember that the actual 
vector or tensor has existence independent of any coordinate system. 

A) Matrix addition, multiplication by a scalar, and subtraction 

Matrices may be added only if both the row dimension (m) and their 
column dimension (n) are equal (such matrices are said to be “of the 
same order”). The addition is accomplished simply by adding each el- 
ement of one matrix to the corresponding element of the other matrix. 
For example: 


5 2 
-3 0 


+ 


-3 1 
0 4 


2 3 
-3 4 
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or in general 


an 

«12 

+ 

' 6ll 

&12 


«21 

022 


&21 

&22 



ail + fell 
(I21 + b-2i 


a 12 + &12 
a 22 + &22 


Note that the result is a matrix of the same order as the matrices being 
added. Note also that matrix addition is commutative, so [A]+[B]=[B] + [A]. 

Multiplication of a matrix by a scalar is straightforward; you simply 
multiply each element of the matrix by the scalar. Thus 


3 A 


' 3 

-2 ' 


' 9 

-6 ' 

1 

0 

1 

3 

0 


and generally 


an 

«i2 


kan 

<3 

021 

022 


ka 2 i 

ka 22 \ 


You can use the rules for addition of matrices and scalar multiplication 
to see that subtraction of matrices is accomplished simply by subtracting 
the corresponding elements. Thus [A] - [B] = [A] + (- 1 ) [B] . So if 


and 


A = 


a n «12 
«21 «22 


B = 


fell &12 

&21 b 2 2 


then 


on + (— 1)&11 

ai2 + (— I)&i2 


on — fell 

Ol2 — &12 

021 + (—'4)621 

022 + (— 1)622 


021 — 621 

022 — b 2 2 


Just as for addition, subtraction of matrices only works for matrices of 
the same order. 


B) Matrix multiplication 

There are several different ways to multiply matrices; the most common 
and most relevant to the vector and tensor concepts in SGVT is to 
multiply two matrices (call them A and B) by multiplying the elements 
of each row in A by the elements of each column of B and then summing 
the results. So if matrix A is given by 

(111 «12 «13 

« 21 «22 «23 
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and matrix B is given by 


&11 

&12 

&13 

621 

&22 

&23 

&31 

&32 

^33 


then the product A times B is given by 


_ 




’ b n 

bl2 

bl3 

an 

012 

ai3 

X 

&21 

&22 

&23 

a 2 i 

£122 

023 


&31 

&32 

&33 . 


_ Oll&ll + Ol2&21 + 013^31 Oll&12 + fll2&22 + Ol3&32 ail&13 + «12^23 + «13^33 
021&11 + a 22&21 + 023^31 a 21&12 + <222^22 + £*23^32 «21&13 + 022^23 + a 23^33 

This result is achieved by multiplying the elements of the first row of 
A by the elements of the first column of B, summing the results, and 
placing the sum in the first row and first column of the result matrix, 
as shown in Figure 1.1. 


Sum products 


Multiply first row 
by first column 

b„ ( ) 




.( ) ( ) 


Place result in first row 
and first column 


a „ b „ +a ,2 b 2 , + a ,3 b 3, 


( ) 


( ) 


( ) 
( ) 


Figure 1.1 Multiplying first row by first column 

The next step is to multiply the elements of the first row of A by the 
elements of the second column of B, summing the products, and placing 
the sum in the first row and second column of the result matrix, as 
shown in Figure 1.2. 

After multiplying the first row of A by the third column of B and placing 
the sum in the first row and third column of the result matrix, the 
same procedure is done with the second row of A - those elements are 
multiplied by the first column of B, summed, and placed in the second 
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Sum products 


Multiply first row 
by second column 


.( ) ( ) 



Place result in first row 
and second column 


a „ b ,2+ a , 2 b 22 +a ,3 b 32 O' 


( ) 


Figure 1.2 Multiplying first row by second column 


Multiply second row 
by first column 



Figure 1.3 Multiplying second row by first column 


row and first column of the result matrix, as shown in Figure 1.3. 

Notice that the matrix that results from multipling a 2x3 matrix (A) by 
a 3x3 matrix (B) is a 2x3 matrix - the result has the number of rows of 
the first matrix and the number of columns of the second matrix. You 
should also note that this type of matrix multiplication only works when 
the number of columns of the first matrix (3 in the case of A) equals the 
number of rows of the second matrix (also 3 in the case of B). 

To understand why matrices are multiplied in this way, consider the 
sorting of five types of toy marbles (Bowlers, Cat’s Eyes, Steelies, Aggies, 
and Commons) into four sizes of package (Small, Medium, Large, and 
Extra Large). The number of each type of marble in each size package 
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is shown in the following table: 

Bowlers Cat’s Eyes Steelies Aggies Commons 

Small package 1 3 2 5 4 

Medium package 2 5 3 7 6 

Large package 5 10 5 12 10 

Extra Large package 10 15 12 16 14 

This array of numbers can be put into a matrix - call it l 'P” the package 
matrix: 

' 1 3 2 5 4 

2 5 3 7 6 

5 10 5 12 10 

. 10 15 12 16 14 . 

Now imagine three containers (Cans, Boxes, and Crates) with different 
numbers of packages in each type: 

Small Pkgs Medium Pkgs Large Pkgs Extra-Large Pkgs 

Can 10 8 5 2 

Box 15 12 7 3 

Crate 40 25 15 10 

Put this array into a matrix called “C” the container matrix: 

' 10 8 5 2 

C = 15 12 7 3 

_ 40 25 15 10 . 

If you wish to find the number of each type of marble in each type of 
container, you could do it like this: 

Bowlers per Can = 10 small packages x 1 Bowler per small package 
+ 8 medium packages x 2 Bowlers per medium package 
+ 5 large packages x 5 Bowlers per large package 
+ 2 extra-large packages x 10 Bowlers per extra-large package 
= (10)(l)+(8)(2)+(5)(5)+(2)(10)=71 Bowlers per Can 
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Likewise, you can find the number of Cat’s Eyes per can using 

Cat’s Eyes per Can=10 small packagesx3 Cat’s Eyes per small package 
+ 8 medium packages x 5 Cat’s Eyes per medium package 
+ 5 large packages x 10 Cat’s Eyes per large package 
+ 2 extra-large packages x 15 Cat’s Eyes per extra-large package 
= (10)(3)+(8)(5)+(5)(10)+(2)(15)= 150 Cat’s Eyes per Can 


And if you wished to find the number of Bowlers per Box, you could use 


Bowlers per Box = 15 small packages x 1 Bowler per small package 
+ 12 medium packages x 2 Bowlers per medium package 
+ 7 large packages x 5 Bowlers per large package 
+ 3 extra-large packages x 10 Bowlers per extra-large package 
= (15)(l)+(12)(2)+(7)(5)+(3)(10)=104 Bowlers per Box 


If you compare the numbers in these calculations to the values in the P 
and C matrices, you’ll see that in each case you’re multiplying the row 
elements of C by the column elements of P, which is exactly how matrix 
multiplication works: 


CP = 


10 8 5 2 

15 12 7 3 

40 25 15 10 


1 3 2 5 4 

2 5 3 7 6 

5 10 5 12 10 

10 15 12 16 14 


71 

104 

265 


150 

220 

545 


93 

137 

350 


198 

291 

715 


166 

244 

600 


So the product matrix shows you how many of each type of marble are 
in each type of container: 



Bowlers 

Cat’s Eyes 

Steelies 

Aggies 

Commons 

Can 

71 

150 

93 

198 

166 

Box 

104 

220 

137 

291 

244 

Crate 

265 

545 

350 

715 

600 


In addition to showing why the rows of the first matrix are multi- 
plied by the columns of the second matrix, this example also illustrates 
the point made earlier: when you multiply two matrices, the number of 
columns in the first matrix must equal the number of rows in the sec- 
ond matrix. So in this case, multiplication works only if you multiply C 
by P, not P by C. The larger point is that matrix multiplication is not 
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commutative, so even if you’re able to multiply two matrices in either 
order (which you can do, for example, with square matrices), the answer 
is not the same. So in matrix world, AB is in general not equal to BA. 

Another difference between matrix multiplication and multiplication 
of numbers is that it’s possible to get zero as the result of multiplying 
two matrices, even if neither matrix is zero. For example, 


2 1 
0 0 


1 0 

-2 0 


0 0 
0 0 


If you study vectors and tensors, you’re likely to come across the mul- 
tiplication of a row vector times a column vector or vice versa, and you 
should understand that such products use the same process described 
above for matrix multiplication, although that may not be immediately 
obvious when you look at the result. For example, multiplying a row 
vector A by a column vector B gives 


[ ail 


ai2 ai3 


bu 
&21 
63 1 


[aii&n + ai2&2i + 013631] . 


This scalar result (essentially the dot product between A and B) comes 
from multiplying the first (and only) row of A by the first (and only) 
column of B and adding the sums. Likewise, if A is a column vector and 
B a row vector, the product AB is 


an 


an&n 

an&w 

aii6i 3 

021 

X [ 6n 612 613 ] = 

a21&n 

021&12 

021613 

a 31 


a 3 i 6 n 

031612 

031613 


Once again, the same rule of matrix multiplication has been applied. 
The first row of A (just an in this case) is multiplied by the first column 
of B (just 6n in this case), and since there are no other elements in the 
first row of A or the first column of B, there is nothing to add, and 
the result (on6n) is written in the first row and column of the result 
matrix. Then the first row of A (again, only an) is multiplied by the 
second row of B (only 612), and the result is written in the first row, 
second column of the result matrix. After doing the same for the first 
row of A and the third column of B, you then multiply the second row 
of A (which is just 021 in this case) by the first column of B (again just 
6n) and write the result in the second row, first column of the result 
matrix. So although you get a scalar when you multiply a row vector by a 
column vector (sometimes called the “inner product” of the matrics) and 
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a matrix when you multiply a column vector by a row vector (sometimes 
called the “outer product” of the matrices), the process is exactly the 
same in both cases. 

Although matrix multiplication is not commutative (so AB is not 
necessarily equal to BA), matrix multiplication is the associative and 
distributive over addition, so for matrices A, B, and C 

(AB)C = A(BC) 

and 

A(B + C) = AB + AC 

as long as you remember not to reverse the order of any of the products. 

You may be wondering if there’s ever a need to multiply each element 
of one matrix by the corresponding element of an equal-size matrix. 
There certainly is (for example, when applying a two-dimensional fil- 
ter funtion to an image). This process is called “element-by-element” 
or “entrywise” matrix multiplication and the result is sometimes called 
the “Hadamard product.” In MATLAB, such multiplication is denoted 
A.*B, where the decimal point before the multiplication symbol signifies 
element-by-element multiplication. 

C) Transpose and trace of a matrix 

The transpose of a matrix is accomplished by interchanging the rows 
with the columns of that matrix. This is usually denoted by placing the 
superscript “T” after the matrix name, so the transpose of matrix A is 
written as A T (MATLAB uses A'). So if the matrix A has elements 

^ _ an ai2 ai3 014 

021 O 22 «23 ®24 

then the transpose of A is given by 

Oil »21 

J^T _ °12 022 

013 023 

. 014 024 

So (. Aij) T = Ajf, notice that the indices have been switched. The trans- 
pose of the product of two matrices is equal to the transpose of each 
matrix multiplied in reverse order, so 

(AB) t = B t A t . 
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so long as the dimensions of the matrices allow such products to be 
formed. 

For square matrices, the trace of the matrix is given by the sum of the 
diagonal elements. The trace is usually denoted by “Tr” so the trace of 
matrix A is written as Tr(A). So if the matrix A is given by 


Oil 

012 

Ol3 

O14 

021 

022 

023 

»24 

031 

032 

033 

O34 

. 041 

O42 

043 

O44 . 


the trace of A is 

Tr{A) = an + a 22 + 033 + 044 = Ojj. 


D) Determinant, minors, and cofactors of a matrix 


The determinant of a square matrix is a scalar calculated by multiplying, 
adding, and subtracting various elements of the matrix. If A is a 2x2 
matrix 


A = 


an CJ12 
O 21 022 


the determinant of A (denoted using vertical bars on each side of A) is 
found by cross-multiplying the upper-left element times the lower-right 
element and then subtracting the product of the lower-left element times 
the upper-right element: 


\A\ = 


Hence if A is 


an CJ12 
«21 022 


A = 


— On 022 — 021012- 


4 -2 
3 1 


the determinant of A is 


\A\ = 


4 -2 
3 1 


= (4) (1) — (3)(— 2) = 4 — (—6) = 10. 


To find the determinant of higher-order matrices, you must use the 
“minors” and “cofactors” of the determinant of the matrix. The minor 
for each element of the determinant of a square matrix is found by 
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eliminating the entire row and the entire column in which the element 
appears and then writing the remaining elements as a new determinant, 
with row and column dimensions reduced by one (so the minor of each 
element of a 3x3 determinant is a 2x2 determinant). So for the 3x3 
determinant of matrix A 



Oil 

Ol2 

O13 

A| = 

021 

022 

023 


O31 

O32 

033 


the minor of element an is 


— 022 
— 032 


023 

“33 


022 023 

032 033 


obtained by elminating the first row and column of the determinant of 
matrix A (since the element for which this minor is taken (on) is in 
the first row and the first column). Likewise, the minor for element 012 
is found by eliminating the row and column in which 012 appears (first 
row and second column): 









021 

031 

— 023 

— a 33 

= 

021 

031 

a 23 

033 


For element 022 , the minor is 


On 

— Oi3 


On 

Ol3 

031 

— a 33 


031 

033 


Before you can use minors to find the determinant of a higher-order 
matrix, you have turn each minor into a “cofactor.” The cofactor of an 
element is just the minor of that element multiplied by either +1 or -1. 
To know which of these to use, just determine whether the sum of the 
element’s row index and its column index is even or odd. If even (as it 
is, for example, for the element in the first row and first column, since 
1+1=2 which is even), the cofactor equals the minor of that element 
mutiplied by +1. If odd (for example, for the element in the first row 
and second column, since 1+2=3 which is odd) the cofactor equals the 
minor of that element multiplied by -1. So for element a»j with row index 
of i and column index of j, 

Cofactor of (oy) = (— 1)^* +J ^ Minor of (a^). 
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So for the minors shown above, the cofactors are 


Cofactor of (an) = (— 1)( 1+1) 

Cofactor of ( 012 ) = (— 1) ( - 1+2 ' 
and 

Cofactor of ( 022 ) = (— 1) (2+2 ^ 


a 22 023 

032 033 

021 023 

031 033 

Oil O13 
031 O33 


= (1)(022033 — 032023) 


= (— l)( 02 l 033 — 031023) 


— (I)( 0 ll 033 — 031013). 


With cofactors in hand, the determinant of a 3x3 matrix is straight- 
forward to find. Simply pick one row or column, and multiply each of 
the elements of that row or column by its cofactor and sum the results. 
So, for example, if you choose to use the first row of the 3x3 matrix A, 
the determinant is given by 


A\ = on(Cofactor of on) + 012 (Cofactor of 012 ) + 013 (Cofactor of ai 3 ). 


or 

| -A | = 0n(l)(022033 — 032023)-|-ai2(— 1) (021033 — 03i023)+0i3 (1) (021032— 031022) • 

The value of the determinant would have been the same if you had used 
a different row (or any of the columns) of matrix A, as long as you 
multiply each element in your selected row or column by its cofactor 
and sum the results. 

The same approach can be used to find the determinant of 4x4 and 
higher-order matrices; you just have to expand each of the determinants 
using elements and cofactors until you get down to the 2 x 2 level. Of 
course, the calculation becomes a bit tedious, so it’s worth your time to 
learn to find determinants using a program like MATLAB. 

Some useful characteristics of determinants are that the determinant 
of the transpose of a matrix equals the determinant of the matrix (so 
\A T \ = \A\) and the determinant of the product of two matrices is the 
same as the determinant of the reverse product (so \AB\ = |-BA|) pro- 
vided that the dimensions of A and B allow both of these products to 
be made. 

You may also find it useful to know that if a matrix has two identical 
rows or columns, or if one row or column is an integer multiple of an- 
other row or column, the determinant of that matrix must be zero. 
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E) Inverse of a matrix 

As indicated in Section C of this document, matrix multiplication is 
quite different from ordinary multiplication of numbers. One example of 
this is that the product of two matrices may be zero even if both of the 
matrices are non-zero (so you can’t simply divide both sides of a matrix 
equation such as AB = 0 by A to get B = 0). Additionally, the matrix 
equation AB = AC does not mean that matrix B equals matrix C (so 
dividing both sides of this equation by A does not work for matrices). 

Differences such as this suggest that matrix division has little in com- 
mon with ordinary division of numbers. About the closest you can get 
to a process similar to division for matrices comes about by considering 
the matrix equation 

AX = B 

where A, X , and B are all matrices. If you wish to find matrix X from 
this equation, here’s one thing you definitely cannot do: 

AX _ B 
~A ~ A 

x ~i 

because this type of division does not work for matrices. 

What you can do to find X is this: you can try to find a matrix A -1 
(called the “inverse” or “reciprocal” of A) with the following property: 

A~ 1 A = I 

where I represents the identity matrix (ones on the diagonal and zeroes 
everywhere else). Now you can find X rather easily: 

AX = B 

A _1 (AA) =A~ 1 B 
(A _1 A)X = A~ 1 B 
IX = A _1 B 
X = A _1 B 

since IX = X (multiplying the identity matrix times any matrix does 
not change that matrix). 

So although you haven’t really divided matrix B by matrix A, you’ve 
used the matrix equivalent of multiplying by the reciprocal of a number 
to achieve the same result as dividing. 
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The question is, does every matrix have an inverse (that is, another 
matrix with which it multiplies to give the identiry matrix), and if so, 
how do you find it? The first part of this question is easy to answer: 
any matrix that has a non-zero determinant (|A| ^ 0) has an inverse. 
Matrices that have no inverse ((|A| = 0 ) are called “singular” matrices, 
and matrices that have an inverse are called “non-singular” matrices. 

But if a certain matrix has an inverse, how do you find that inverse? 
There are several ways to go about this, but one approach is to use 
the concepts of cofactors, transpose, and determinant described above. 
Those concepts appear in the following equation for the inverse of matrix 
A: 

x (Matrix of cofactors of A) T 


in which the “Matrix of cofactors of A” is a matrix in which each element 
of A is replaced by that element’s cofactor. 

Here’s how that works for a 3 x 3 matrix. If matrix A is the usual 



Oil 

012 

O13 

A = 

021 

022 

023 


031 

032 

033 

then the matrix of cofactors (MoC) 

for A 

is 


MoC ( A ) 


022033 — O32O23 
(012033 — 032013) 
012 023 — 022013 


(021033 — 031023) 
011033 — 031013 
(0n023 — 021013) 


021032 — 031022 

(011032 — 031012) 
011022 — 021012 


The equation for the inverse of A shown above requires that you take 
the transpose of this matrix, which is 

[MoC (A)] t = 

Dividing this matrix by the determinant of A provides the inverse of A. 

Fortunately, the inverse of non-singular matrices is easily found using 
most scientific calculators or computers, usually by defining a matrix A 
and then raising A to the power of minus one. 

For a diagonal matrix, the inverse is simply another diagonal matrix 
in which each diagonal element is the reciprocal of the corresponding 


O22O33 — O 32(123 
(021033 — 031023) 
O21O32 — 031022 


(012033 — 032013) 
011033 — 031013 

(011032 — 031012) 


«12«23 — 022013 
( 0 ll 023 — 021013) 
011022 — 0 2 lOl2 
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element in the original matrix. So 


Oil 

0 

0 

-1 

1 

an 

0 

0 

0 

022 

0 

= 

0 

1 

a 22 

0 

0 

0 

033 . 


0 

0 

1 

0-33 _ 


F) Simultaneous linear equations and Cramer’s Rule 

The relationship between matrices and simultaneous linear equations 
can be understood by considering equations such as 

2x + 5 y — z = 12 
—3* — 3y + z = — 1 
x + y = 1 . 

Using the rules of matrix multiplication, this system of equations can be 
written as a single matrix equaton: 


2 

5 

-1 ' 


X 


12 

-3 

-3 

1 

X 

y 

= 

-1 

1 

1 

0 


z 


1 


In general, three linear equations in three unknowns (aq, X 2 , * 3 ) can be 
written as 

anaq + 012X2 + 013X3 = &i 
®2lXl + 022X2 + 023X3 = &2 
O31X1 + 032X2 + O33X3 = &3 


Oil 

012 

Ol3 


’ Xl 


' h - 

021 

022 

023 

X 

X 2 

= 

b 2 

031 

032 

033 . 


. x 3 . 


63 


If you define the matrices 


Oil 

O12 

Ol3 

021 

022 

023 

. O31 

O32 

033 . 


’ Xi 


X = 

x 2 



. *3 . 



and 
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and 


b = 


then the system of equations can be written as a single matrix equation: 


Ax = b 


which can sometimes be solved using the inverse of matrix A: 

x = A~ 1 b. 

The reason that the word “sometimes” appears in the previous sentence 
is that there are several conditions which will prevent this approach from 
succeeding. For example, the system of equations may be inconsistent - 
that is, there may be no set of values for xi,X2,x 3 that satisfy all three 
equations. In that case, you will not be able to find the inverse of A 
because it will be a singular matrix. And if matrix b equals zero, you 
have a system of homogeneous linear equations, which means there will 
be only the trivial solution ( x\ = X2 = £3 = 0) if A is non-singular or 
an infinite number of solutions if A is singular. 

In cases for which A is non-singular and b does not equal zero, you 
can find the values of aq, X2, and x 3 by finding the inverse of matrix 
A and multiplying that inverse by matrix b, or you can use Cramer’s 
Rule. In that approach, the first unknown (aq in this case) is found by 
replacing the values in the first column of the coefficient matrix (A) with 
the elements of matrix b and dividing the determinant of that matrix by 
the determinant of A. Here’s how that looks: 


bi 

012 

ai3 

b 2 

022 

023 

b 3 

a 32 

a 33 

on 

012 

a 13 

021 

022 

«23 

031 

032 

a33 


Likewise, to find the second unknonwn ( aq in this case), replace the 
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values in the second column of A with the elements of b: 


on 

b i 

Ol3 

021 

b 2 

023 

031 

bz 

033 

an 

Ol2 

«13 

021 

022 

«23 

a 3i 

032 

«33 


and to find x$ use 


Oil 

«12 

bi 

021 

022 

^2 

031 

032 

h 

an 

012 

013 

021 

022 

023 

O31 

032 

033 


Thus for the equations given at the start of this section, 

'2 5 -1 ' 

A= -3-3 1 

.1 1 0 
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so 

_ 12 [(— 3 )( 0 ) - ( 1 )( 1 )] + 5 (— 1 )[(— 1 )( 0 ) - ( 1 )( 1 )] - 1 [(- 1 )( 1 ) - ( 1 )(— 3 )] 
■ 2 [( 3 ) ( 0 ) - ( 1 )( 1 )] + 5 (— 1 )[(— 3 )( 0 ) - ( 1 )( 1 )] - 1 [( 3 ) ( 1 ) - ( 1 )(— 3 )] 

_ -12 + 5-2 _ Q 
- 2 + 5-0 

Proceeding in the same way for y and z gives 

2 12 -1 
-3 -1 1 

1 1 0 


-3 -3 1 

1 1 0 


so 


and 


-2 + 12 + 2 
- 2 + 5-0 


2 5 12 

-3 -3 -1 
1 1 1 


-3 -3 1 

1 1 0 


so 

-4 + 10 + 0 0 

z = = 2. 

- 2 + 5-0 


G) Matrix diagonalization using eigenvectors and eigenvalues 

In the matrix equation discussed in the previous section 

Ax = b 


the matrix A operates on matrix x to produce matrix b. If x and b 
are vectors, this equation represents the transformation by matrix A 
of vector x into vector b. In some cases, the operation of A on the 
components of x produces the components of a vector b that is a scaled 
(but not rotated) version of x. In such cases, the equation becomes 


Ax = Ax 
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where A represents a scalar multiplier (and scalar multipliers can change 
the length but not the direction of a vector) . 

Any vector x which satisfy this equation for a matrix A is called an 
“eigenvector” of matrix A, and the scalar A is called the “eigenvalue” 
associated with that eigenvector. 

The eigenvalues of a matrix can be very useful in finding a diagonal 
version of that matrix, so you may wish to understand how to find the 
eigenvalues of a given matrix. To do that, write the previous equation 
as 

Ax — Xx = 0 


which, since lx = x, can be written as 

Ax — A (lx) = 0 


or 

(A - \I)x = 0. 

which means that either x = 0 (which is a the trivial case) or 

\A — XI\ =0. 


This equation is called the “characteristic equation” for matrix A, and 
for a 3x3 matrix it looks like this: 


Oil O12 

«13 


1 

0 

021 »22 

023 

- A 

0 

1 

031 032 

033 


0 

0 

an — A 

O12 

O13 

021 

022 " 

-A 

023 

031 

O32 

O33 — A 


This expands to 

(dll — A)[(d 22 — A)(d 33 — A) — <232023] 

+ 012 ( 1 ) [021 (033 — A) — <131(123] 
+ 013 [021O32 — <231(022 — A)] = 0 . 


Finding the roots of this polynomial provides the eigenvalues (A) for 
matrix A , and substituting those values back into the matrix equation 
Ax = Ax allows you to find eigenvectors corresponding to each eigen- 
value. The process of finding the roots is less daunting than it may 
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appear, as you can see by considering the following example. For the 
3x3 matrix A given by 


A = 

the characteristic equation is 

4- A 
-7 
5 


4 -2 -2 

-7 5 8 

5 -1 -4 


-2 -2 
5- A 8 
-1 -4 -A 



or 


(4 — A) [(5 — A) (—4 — A) — (— i)(8)] 

— 2(— 1)[(— 7)(— 4 — A) — (5)(8)] 

— 2[(— 7)(— 1) — (5)(5 — A)] = 0. 


Multiplying through and subtracting within the square brackets makes 
this 

(4 - A)(A 2 - A - 12) + 2(7A - 12) - 2(5A - 18) = 0 


or 

-A 3 + 5A 2 + 12A - 36 = 0. 


Finding the roots of a polynomial like this is probably best left to a 
computer, but if you’re lucky enough to have a polynomial with integer 
roots, you know that each root must be a factor of the term not involving 
A (36 in this case). So (+/-) 2, 3, 4, 6, 9, 12, and 18 are possibilities, and it 
turns out that +2 works just fine: 

— (2) 3 + 5(2 2 ) + 12(2) - 36 = -8 + 20 + 24 - 36 = 0. 


So you know that one root of the characteristic equation (and hence one 
eigenvalue) must be +2. That means you can divide a factor of (A — 2) 
out of the equation and try to see other roots in the remainder. That 
division yields this: 


— A 3 T 5A" T 12A — 36 

(A = 2) 


—A 2 + 3A + 18. 


The roots remaining polynomial on the right-hand side of this equation 
are +6 and -3, so you now have 


-A 3 + 5A 2 + 12A - 36 = (A - 2)(6 - A)(A + 3) = 0. 
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So matrix A has three distinct eigenvalues with values +6, - 3 , and +2; 
these are the factors by which matrix A scales its eigenvectors. You 
could find the eigenvectors of A by plugging each of the eigenvalues 
back into the characteristic equation for A , but as long as you can find 
N distinct eigenvalues for an NxN matrix, you can be sure that A can 
be diagonalized simply by constructing a new diagonal matrix with the 
eigenvalues as the diagonal elements. So in this case, the diagonal matrix 
(call it A') associated with matrix A is 


A' = 


6 0 0 ' 
0-3 0 
0 0 2 


To see why this is true, consider the operation of matrix A on each of 
its three eigenvectors (call them e, /, and g ): 


Ae = Are 
Af = A 2 f 
Ag = A 3 g 


Now imagine a matrix E whose columns are made up of the eigenvectors 
of matrix A: 



ei 

/1 

51 ’ 

E = 

e2 

h 

52 


. e 3 

S 3 

53 . 


where the components of eigenvector e are (ei,e2,e3), the components 
of eigenvector / are (fi, f 2, f 3), and the components of eigenvector g are 
(51,52, 53)- Multiplying matrix A (the original matrix) by E (the matrix 
made up of the eigenvectors of A), you get 



Oil 

012 

O13 


’ ei 

fi 

5i ’ 

AE = 

021 

022 

023 

X 

e2 

Si 

52 


. O31 

032 

033 . 


. e 3 

S3 

93 . 


which is 


AE = 


anei + 01262 + (11363 
02161 + 02262 + 02363 
03161 + a 32 e 2 + 03363 


an/i + 012/2 + 013/3 

021/1 + 022/2 + 023/3 

031/1 + 032/2 + 033/3 


01151 + 01252 + 01353 
02151 + 02252 + 02353 
«3l5l + 03252 + O3353 


The columns of this AE matrix are the result of multiplying A by each 
of the eigenvectors. But you know from the definition of eigenvectors 
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and eigenvalues that 

ei Aiei 

Ae = Aie = Ai e2 = Aie2 

. e 3 J L ^l e 3 

and 

fi A2/1 

Af = A2 / = A2 /2 = A2/2 

. /3 . . ^2/3 . 

and 

Si ^3^1 

^5 = A3 g = A3 g2 = A3<?2 

. S 3 J L ^3 S 3 . 

This means that the product AE can be written 

Aiei A2/1 A 3 < 7 i 

ydi? = Aie2 A2/2 A3^2 

. Aie 3 A2/3 A 3<73 

But the matrix on the right-hand side can also be written like this: 

Aiei A2/1 A3gi ei /1 g \ Ai 0 0 

Aie2 A2/2 A3</2 = e2 /2 S2 x 0 A2 0 

. Aie3 A2/3 A3g3 J |_ e 3 /3 S 3 J L 0 0 A3 _ 

This means that you can write 

' Ai 0 0 

AE — E 0 A2 0 

.0 0 As . 

and multiplying both sides by the inverse of matrix E ( E -1 ) gives 


E~ l AE 


Ai 

0 

0 


Ai 

0 

0 

0 

A2 

0 

— 

0 

A2 

0 

0 

0 

CO 


0 

0 

A3 _ 


If you tracked the discussion of the similarity transform in Section 6.1, 
you’ll recognize the expression E _1 AE as the similarity transform of ma- 
trix A to a coordinate system with basis vectors that are the columns 
of matrix E. Those columns are the eigenvectors of matrix A, and the 
matrix that results from the similarity transform (call it A') is diagonal 
and has the eigenvalues of A as its diagonal elements. 
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